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Abstract 


Geospatially  enriched  synthetic  environments  are  needed  for  development  and 
assessment  of  unmanned  ground  vehicle  (UGV)  performance  to  support  sensor  fusion 
and  sense  making.  This  work  will  address  how  the  high- fidelity/resolution  environment 
is  achieved  and  integrated  to  inform  simulations  addressing  critical  questions.  We  will 
investigate  a  multi-resolution  modeling  capability  to  inform  development  of  a  high- 
fidelity  synthetic  environment  (HFSE)  testbed  and  to  link  to  other  models  and 
simulations.  This  report  will  discuss  results  and  lessons  learned  in  developing  an 
engineering-  and  operational-level  experiment  for  proof  of  concept. 
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Chapter  1 :  Introduction 


Ground  robotic  systems  are  integral  components  of  the  current  and  future  forces, 
integrated  across  a  variety  of  missions  including  reconnaissance,  search  and  detection, 
fire  support,  force  protection,  and  logistics.  As  advancements  are  made  along  the 
spectrum  toward  the  goal  of  autonomous  navigation,  enriched  geospatial  synthetic,  or 
simulated,  environments  can  provide  a  means  to  assess  system  and  subsystem 
performance.  Providing  such  environments  will  promote  commonality  and  comparison 
of  experimental  results,  will  reduce  costs,  and  will  speed  development. 

The  U.S.  Army  Engineer  Research  and  Development  Center  is  leading  an  effort 
with  partners,  including  the  United  States  Military  Academy  (USMA),  to  create  a  high 
fidelity  synthetic  environment  (HFSE)  testbed  to  support  development  of  unmanned 
ground  vehicles  (UGV),  including  engineering-  and  operational-level  capabilities. 
Engineering-level  capabilities  focus  on  system  and  subsystem  performance  where  as 
operational-level  capabilities  are  at  higher  echelons  and  include  examining  contributions 
of  UGV  systems  to  an  operational  mission. 

A  stakeholder  analysis  was  conducted  (Goerger,  Moore  and  Nagle,  2008)  and 
identified  the  need  for  a  HFSE  to  facilitate  UGV  performance  simulation  and  analysis, 
particularly  in  the  areas  of  mobility,  obstacle  and  target  detection,  and  navigation. 
Stakeholders’  needs  for  a  HFSE  are  summarized  as  the  following:  (a)  interfaces  or 
integration  that  are  simple  to  achieve  and  will  not  require  retrofitting  or  reprogramming; 
(b)  near-real-time  run-time  performance;  (c)  support  exercises  and  experiments,  and  (d) 
good  visual  representation  and  correlation. 
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Based  on  the  results  of  the  stakeholder  analysis,  an  integrated  decision  analysis 
framework  was  developed  to  assess  HFSE  testbed  design  alternatives  (Goerger  et  al. 
2008).  The  decision  framework  linked  functionality  parameters,  value  models,  and 
metrics  to  generate  and  assess  design  alternatives.  Assessing  how  well  the  HFSE 
supports  stakeholder  values  regarding  ease  of  interfaces/integration,  run-time 
performance,  support  to  exercises  and  experiments,  and  visual  representation  is  the  key  to 
the  development  of  a  HFSE. 

To  inform  development  of  a  HFSE  testbed  and  to  link  to  other  models  and 
simulations,  a  major  thrust  of  the  effort  presented  in  this  report  consists  of  developing 
and  conducting  experimental-  and  operational-level  experiments.  This  report  describes 
the  experimental  environment,  or  testbed,  that  was  created  and  the  experiments 
conducted  to  investigate  means  of  meeting  stakeholders’  needs  and  collects  lessons 
learned. 

Background 

In  2006,  the  Office  of  the  Secretary  of  Defense  (OSD)  Joint  Ground  Robotics 
Enterprise  (JGRE)  funded  the  development  of  the  Virtual  Autonomous  Navigation 
Environment  (VANE).  A  stakeholder  analysis  and  needs  analysis  was  conducted  and  an 
initial  set  of  use  cases  were  developed.  This  project  takes  the  information  that  was  gained 
during  these  projects  and  applies  it  to  the  development  of  a  multi-resolution  synthetic 
environment  to  facilitate  the  assessment  of  UGVs  in  a  high  fidelity  synthetic 


environment. 


1.2 


Use  Case  Development 

To  help  guide  the  experimentation  process  in  the  development  of  an  HFSE,  an 
initial  set  of  use  cases  was  developed  (Nagle,  Goerger,  and  DeLong,  2008).  The  use 
cases  were  based  on  an  analysis  of  critical  capability  gaps  identified  in  the  U.S. 
Army/U.S.  Marine  Corps  Ground  Robotics  Master  Plan  (GRMP)  (Robotics  Systems  Joint 
Project  Office  (RS  JPO),  2007). 

In  Nagle  et  al.  (2008),  a  use  case  was  described  as  a  specific  scenario,  or  vignette, 
in  which  a  UGV  is  employed  to  contribute  to  a  mission.  It  is  a  narrative  description  of  a 
sequence  of  actions  a  war  fighter  equipped  with  a  UGV,  would  undertake  to  accomplish  a 
goal.  The  Use  Cases  developed  do  not  identify  requirements,  but  rather  imply  them  in 
the  stories  they  tell,  leaving  it  up  to  an  analyst  to  identify  the  requirements.  The  use  cases 
avoid  identifying  or  describing  specific  platforms,  but  rather  create  opportunities  for 
analysts  to  identify  capabilities.  The  capability  requirements  derived  in  this  manner 
provided  a  valuable  supplement  to  UGV  capability  gaps  assessments  which  are  explored 
through  a  series  of  experiments. 

The  experimentation  process  is  two-fold.  First,  a  UGV  experimental  testbed  is 
created  and  evaluated  through  an  engineering-level  experiment  based  on  the  stakeholder 
analysis  completed  with  respect  to  the  HFSE  while  applying  a  derived  use  case.  There 
were  several  recommendations  from  the  stakeholder  analysis  as  detailed  in  Goerger,  et 
al.,  2008.  Each  of  these  recommendations  is  taken  into  consideration  during  the 
development  of  the  UGV  experimental  testbed 

Second,  the  UGV  experimental  testbed  is  used  to  evaluate  the  use  case  for  the 
operational-level  experiment  in  order  to  demonstrate  the  capabilities  of  the  testbed  with 
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respect  to  the  actual  performance  of  the  UGV  given  the  use  case  in  the  HFSE.  For  this 
experiment,  Use  Case  #1  -  Locate  Possible  Enemy  Improvised  Firing  Point/Device,  is 
utilized  to  demonstrate  some  basic  engineering-level  capabilities  and  operational-level 
UGV  functionality  within  the  testbed.  Measures  of  effectiveness  were  developed  for  the 
purpose  of  assessing  both  the  engineering-  and  operational-level  UGV  experimental 
testbed  given  the  Use  Case  #1. 


Chapter  2:  Experimental  Overview 

Figure  1  depicts  the  approach  and  outcomes,  or  products,  for  the  HFSE 
requirements  and  design  analysis  conducted  in  FY07  and  FY08.  The  approach  involved 
conducting  a  stakeholder  analysis,  developing  a  value  hierarchy  and  model,  generating 
and  assessing  the  HFSE  design  options  using  the  value  model,  and  conducting 
experiments  to  demonstrate  the  HFSE  potential  and  proof  of  concept.  The  stakeholder 
analysis  gathered  information  on  issues  and  the  needs  of  stakeholders.  Thus,  as  a 
foundational  part  of  the  development  of  the  HFSE,  a  literature  review  was  conducted  and 
a  series  of  interviews  and  collaborative  sessions  with  stakeholders  were  used  to  elicit 
information  pertinent  to  its  functionality.  This  led  to  the  generation  of  a  framework  for 
assessing  the  HFSE  architecture  and  design  that  mapped  needed  HFSE  functions  to  value 
measures.  Use  Case  development  provides  a  focus  for  scoping  assessments  and  supports 
experiments  that  develop  the  HFSE  within  the  stakeholder  community.  The  constraints, 
limitations,  and  assumptions  for  the  analysis  are  detailed  in  Nagle,  et  al.,  2008. 


4 


StakeholderAnalysis  value  Modeling 


Alternative  Generation 
&  Assessment 


Stakeholder 

Collaborative 

Exercises 


Sponsors’ 
Weighted  Needs 


Design 

Recommendations/ 
Specified  Architecture 


Literature 

Review 


Identify  System 
Functions 


/  /  / 


Architecture 

Brainstorming 


/ 


Experiment  2 


Z 


\ 


\ 


Stakeholder 

Interviews 


User  Needs/ 
Requirements 


Map  Value 
Measures 
Needs 


;  \  \ 

to  \  Develo 


Develop  Use  Cases 


Value  Model  for 
Scoring  Designs 


Experiment  1 


Figure  1.  Tasks  and  products  for  the  high-fidelity  synthetic  environment  (HFSE) 
requirements  and  design  analysis. 

The  next  step  for  this  project  was  to  take  what  was  learned  as  a  result  of  the 
stakeholder  analysis  and  develop  the  UGV  HFSE  testbed  in  order  to  evaluate  the 
capability  as  well  as  evaluate  the  potentials  in  using  this  environment  for  UGV 
assessment.  Chapter  3  details  how  the  UGV  HFSE  testbed  is  developed  and  the 
characteristics  of  interest  that  were  used  during  this  development  process.  Chapter  4 
details  Use  Case  #1  and  the  scenario  developed  to  assess  the  environment  and  the  UGV 
performance.  Chapter  5  assesses  the  testbed  environment  in  an  engineering-level 
experiment  and  Chapter  6  assesses  the  UGV  performance  in  the  environment  in  an 
operational-level  experiment.  Chapter  7  summarizes  the  results  of  this  work  and  Chapter 
8  addresses  future  work  to  be  done  in  this  area. 
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Chapter  3:  UGV  HFSE  Testbed  Development 


The  results  of  the  stakeholder  analysis  identified  the  key  functions  needed  to 
assess  UGV  in  the  HFSE.  Figure  2  depicts  the  functions  and  sub  functions  identified 
during  the  stakeholder  analysis. 


Objective:  Provide  a  high-fidelity,  high-resolution  environment  for 
assessment  of  UGV  systems  and  subsystems  (early  acquisition 
process)  across  concepts,  designs,  and  operations  in  order  to  achieve 
implementation  of  the  best  systems. 
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Environmental 
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Figure  2.  Functional  analysis  of  the  unmanned  ground  vehicle  high-fidelity  synthetic 

environment  testbed. 

These  functions  led  to  the  identification  of  the  necessary  functions  within  the 
testbed  as  well  as  the  metrics  in  which  to  measure  the  capabilities  of  the  testbed.  The 
development  of  the  UGV  HFSE  testbed  centered  on  using  the  ERDC  model  Autonomous 
Navigation  Virtual  Environment  Laboratory  (ANVEL)  for  the  high-fidelity  UGV 
representation  and  the  Army  simulation  One  Semi -Automated  Forces  (OneSAF)  as  the 
operational  level  driver.  This  model  and  simulation  are  connected  via  High  Level 
Architecture  (HLA)  software  provided  by  the  Modeling  Architecture  for  Technology  and 
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Research  Experimentation  (MATREX)  products  (Hurt  et  al.  2006).  OneSAF  is  also  the 
operational  level  driver  for  TRADOC  experimentation  and  is  at  the  core  of  the  Battle  Lab 
Collaborative  Simulation  Environment  (BLCSE)  Federation  (DeLong,  2005).  The 
BLCSE  Federation  utilizes  MATREX  to  facilitate  its  HLA  environment.  Given  the  large 
number  of  stakeholders  familiar  with  this  proven  environment,  it  is  a  natural  conclusion 
to  bring  ANVEL  to  this  environment  where  the  benefits  from  this  research  will  be  far 
reaching  not  just  as  a  proof  of  principle  but  also  to  help  stakeholders  to  start 
experimenting  and  assessing  UGV  performance  in  order  to  further  their  research  goals. 

3.1  ANVEL  Overview 

ANVEL  is  a  rigid-body,  physics  simulation  specialized  for  high-resolution,  real¬ 
time  mobility  modeling  of  wheeled  and  articulated  platforms.  Its  purpose  is  to  represent 
mechanical  system  interactions  with  realistic  movements  while  providing  interfaces  for 
mechanical  systems  and  sensor  models.  ANVEL  also  facilitates  an  easy  configuration  of 
mission  scenarios.  The  ANVEL  architecture  is  described  as  open  computing  framework 
with  vehicle/object  modeling,  rigid  body  dynamics  simulation,  actuator  modeling,  terrain 
generation  and  a  ground  contact  interface.  ANVEL  allows  for  the  simultaneous  viewing 
of  sensor  output,  vehicle  mobility  and  Autonomous  Navigation  System  (ANS)  and 
features  a  mission  rehearsal  and  playback  function.  ANVEL  is  being  developed  for 
compliance  with  the  Joint  Architecture  for  Unmanned  Systems  (JAUS)  which  is  one  of 
the  stakeholder  recommendations. 

Additional  benefits  of  ANVEL  are  the  interaction  with  guest  JAUS  compliant 
subsystems,  faster  debugging  of  components  due  to  the  modularity  of  the  ANVEL  code 
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and  various  viewing  options  for  output  data.  Figure  3  shows  the  high  resolution 
graphics\scene  quality  capability  within  ANVEL. 


Figure  3.  ANVEL  Graphics. 

3.2  OneSAF  Overview 

OneSAF1  is  a  U.S.  Army  constructive  simulation  that  was  developed  to  reduce  the 
number  of  simulations  and  encourage  interoperability  and  reuse  while  meeting  the 
requirements  to  model  and  simulate  the  next  generation  force  requirements.  OneSAF  is  a 
composable,  platform- independent,  entity-level  Computer  Generated  Forces  (CGF) 
simulation  for  a  brigade  and  below  sized  unit,  down  to  the  individual  combatant,  and  is 
capable  of  stimulating  virtual  and  live  simulations.  OneSAF  composition  tools  include 
entity,  unit  and  behavior.  OneSAF  has  the  capability  of  representing  multiple  forces  and 
relationships.  Relationships  between  forces  are  changeable\asymmetric  and  are  portrayed 
as  friendly,  hostile,  suspect,  and  neutral.  Some  crowd  modeling  is  available.  OneSAF 


1  http://www.onesaf.net/community/ 
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models  the  contemporary  operating  environment  to  include  improvised  explosive  devices 
(IEDs),  mouse  hole  creation,  dynamic  side  changes,  reduced  profile  shooting,  detection 
of  vehicle  borne  IEDs,  indirect  fire  weapons  used  as  direct  fire  weapons,  urban 
operations  MEDEVAC,  sniper  tactics,  penetration  of  building  walls,  conduct  raids, 
controlled  mines,  ambush  and  shield  tactics,  expedient  field  fortifications,  decoys  and 
rocket  and  mortar  attacks. 

OneSAF  represents  the  full  range  of  Battlefield  Functional  Areas  (BFA),  systems 
and  operations,  semi-  or  fully-automated  behaviors,  multi-resolution,  validated  models, 
multi-resolution  terrain,  ultra  high  resolution  buildings,  two-way  connectivity  to 
Command,  Control,  Communications,  Computers  and  Intelligence  (C4I)  devices.  It  also 
has  data  collection  and  after  action  review  tools. 

MATREX  Overview 

MATREX  facilitates  the  integration  of  models  and  simulations  with  multiple 
fidelity  levels  in  a  distributed  environment,  for  the  purpose  of  experimentation  and 
analysis.  The  benefits  of  the  MATREX  environment  is  that  it  enables  the  reconfiguration 
and  reuse  of  components  for  engineering  model  development  and  evaluation,  technology 
tradeoffs,  capabilities  assessments,  concept  development,  experimentation  and  testing, 
and  supports  decision  making  over  the  entire  acquisition  lifecycle. 

MATREX  is  an  architecture  based  on  services  that  “exchange  object  data  through  a 
runtime  infrastructure”  (Hurt,  McDonnell  and  McKelvy,  2006).  Interoperability  includes 
HLA  RTI  1.3NG,  IEEE1516,  TENA  5.4,  and  DIS  FY08.  The  primary  elements  of 
MATREX  are  the  architecture\environment,  model  and  simulations,  tools, 
interoperability  and  collaboration  features.  MATREX  is  advancing  simulation 


technology,  infrastructure  and  processes  to  enable  better  informed  decision  making. 
MATREX  provides  many  of  the  tools  and  methodologies  to  help  reduce  technical  costs 
and  schedule  risk.  (Hurt,  et  al.,  2006.) 


Chapter  4:  Experiment  Scenario  Development 


4.1  Overview 

Use  cases  are  developed  in  the  most  general  terms,  and  are  meant  to  broaden  the 
spectrum  of  considered  capabilities  and  technology  enablers  for  UGV.  The  use  cases  are 
intended  to  be  generic  with  no  focus  on  a  particular  foreign  country  or  group  of  people.  The 
use  case  timeframe  is  applicable  for  current  and  future  forces  through  2032  as  identified  by 
the  Office  of  the  Secretary  of  Defense’s  Unmanned  Systems  Roadmap  2007-2032  (Office  of 
Secretary  of  Defense,  2007).  This  experiment  focuses  on  the  employment  of  use  case  #1- 
Locate  possible  enemy  improvised  firing  point/device  (Nagle,  et  al.,  2008).  Maneuver, 
search,  detect  and  locate  tasks  underlies  this  Use  Case. 

4.2  Use  Case  #1:  Locate  Possible  Enemy  Improvised  Firing  Point/Device 

Units  conducting  counter-IED  operations  which  consist  of  UGV  are  employed  to 
identify  IED  and  firing  points  using  the  functional  threads  of  maneuver,  search,  detect, 
and  locate  while  conducting  route  clearing  operations.  Table  1  describes  use  case  #1 
which  was  designed  to  operate  an  UGV  in  a  HFSE  and  perform  maneuver,  search,  detect, 
and  locate  tasks  (Nagle  et  al.,  2008).  The  Use  Case  was  built  around  locating  a  possible 
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enemy  IED  or  firing  point.  This  is  a  scenario  that  could  be  performed  by  autonomous, 
semi-autonomous  or  tele-operated  UGV  using  current  and  future  vehicle  platforms. 


Table  1.  Use  Case  for  locating  a  possible  enemy  improvised  firing  point/device. 

(From  Nagle,  et  al.,  2008) 


Use  Case  #1:  Locate  possible  enemy  improvised  firing  point/device 

Name  Description 

Maneuver,  search,  detect,  and  locate  possible  enemy  improvised 
firing  point/device. 

Situation 

Possible  enemy  improvised  firing  point/device.  Iraq  urban  terrain; 
heavy  civilian  foot  and  vehicle  traffic;  enemy  direct  fire  unlikely; 
daylight  hours. 

Task 

Conduct  reconnaissance  using  UGV. 

Purpose 

Confirm  or  deny  possible  rocket  firing  point. 

Doctrine 

ART  2.3  Conduct  Intelligence,  Surveillance,  and  Reconnaissance. 
ART  2.3.3  Conduct  Tactical  Reconnaissance 

MCT  2.2  Collect  Data  and  Intelligence 

MCT  2.2.1  Conduct  Tactical  Reconnaissance 

Notes 

Maneuver  operations  are  highly  restricted  since  armored  vehicles 
may  damage  streets,  homes,  automobiles,  etc.  Additionally,  streets 
are  often  cluttered  with  day-to-day  activities  that  should  not  be 
interrupted  with  military  operations. 

There  is  much  more  background  clutter  in  the  form  of  radio 
transmissions,  lights,  pedestrians,  civilian  automobiles,  and  other 
interferences  that  degrade  the  performance  of  military 
communications,  sensors,  and  human  sensing. 

The  probability  of  War  fighter  interacting  with  civilians  is  high  so 
there  is  a  much  greater  requirement  for  our  War  fighter  to 
understand  the  local  populace  in  terms  of  language  and  culture. 

Combat  operations  focus  principally  on  a  limited,  concrete  set  of 
effects,  such  as  the  number  of  targets  engaged. 
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4.3  Scenario  Development 


With  Use  Case  #1  in  mind  and  the  maneuver,  search,  detect  and  locate  threads,  an 
IED  scenario  in  a  OneSAF  urban  environment  was  developed.  The  scenario  situation, 
mission  and  execution  are  described  in  the  following  sections. 

4.3.1.  Situation 

4.3. 1.1  Enemy  Forces 

Insurgents  have  been  targeting  road  intersections  along  Abu  Ghrab  Expressway 
(Rasid  AlKaylani  Street)  with  indirect  fire  (60-mm  mortars).  Intelligence  confirms  that 
the  insurgents  are  operating  from  random  locations  within  1  km  south  of  Rasid  AlKaylani 
Street.  IED  are  expected  to  be  used  defensively  in  the  area  surrounding  the  insurgent 
improvised  firing  positions.  Three  to  six  insurgents  are  believed  to  be  in  the  area  of 
operations  (AO)  and  are  equipped  with  60-mm  mortars  and  machine  guns.  Their  most 
probable  course  of  action  (COA)  upon  detection  will  be  to  move  individually  to  the  south 
and  hide  among  the  buildings.  One  or  more  IED  are  suspected  in  the  area  surrounding  the 
insurgent  firing  position. 

4.3. 1.2  Friendly  Forces 

Higher  level  Coalition  headquarter’s  desire  that  the  Abu  Ghrab  Expressway  be 
secured,  recently  deployed  UGV  should  be  utilized  in  platoon  level  patrols,  to  secure  the 
Expressway.  These  patrols  will  have  two  AH-64A-11  Apaches  on  standby  for  air 
support,  and  UGV  teams  will  be  attached. 
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Figure  4.  Area  of  Operations  for  the  Use  Case  Scenario. 


4.3.2.  Mission 

1st  Coalition  Stryker  Recon  Platoon  with  attached  UGV  team  will  conduct  a 
reconnaissance  of  the  AO  BLACKFLY  NLT  TTTTDDMMYYYY  to  conduct  route 
clearing  operations  along  streets  to  the  south  of  the  Abu  Ghrab  Expressway.  The  platoon 
will  move  randomly  along  streets  south  of  the  Abu  Ghrab  Expressway.  Upon  discovery 
of  an  IED  the  area  is  to  be  cordoned  off  and  air  support  called  in. 

4.3.3.  Execution 

The  purpose  of  this  mission  is  to  neutralize  the  insurgents  operating  to  the  south 
of  Abu  Ghrab  Expressway  between  intersections  at  approximately  33°19’40”N, 
44°18’2”E  and  33°19’48”N,  44°19’29”E,  the  intersection  with  ArRabi  Street.  These 
insurgents  have  been  targeting  the  Expressway  with  indirect  fire  and  IED.  At  the  end  of 
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this  mission,  we  would  like  to  have  all  IED  threats  and  insurgents  eliminated  without  the 
effects  of  collateral  damage  within  the  urban  environment.  The  1st  Platoon  will  conduct 
the  route  clearance  operation  with  the  assistance  of  an  UGV.  They  will  conduct  these 
operations  with  air  support  to  ensure  the  safe  transit  of  friendly  units.  Conduct  of  the 
operation  with  UGV  and  Stryker  platoon  moves  in  a  column  south  of  the  Abu  Ghrab 
Expressway  to  the  objective.  Upon  discovery  of  an  IED  and  insurgent  indirect  fire 
positions,  the  platoon  calls  for  air  support.  On  order,  two  AH-64A-11  Apaches  attack 
insurgent  position  along  axis  Blue  Arrow.  If  an  IED  explodes,  the  insurgents  should 
scatter.  If  the  IED  is  detected,  but  not  detonated,  the  UGV  should  report  its  location  and 
continue.  The  Stryker  platoon  should  move  to  positions  which  block  insurgent  escape 
routes  and  call  for  air  support.  Insurgents  may  then  scatter  when  rotary  wing  aircraft 
(RWA)  arrives.  Insurgents  are  found  by  RWA  or  by  the  Stryker  platoon. 


Suspected 
Insurgent  Firing 
Position 

jm - — \ 


Final  leg  of  UGV 
and  Stryker  Platoon 
Route 


-  - 


■■■■■■■■ 


Figure  5.  Scheme  of  maneuver.  Google  Earth  image  of  the  BLACKFLY  AO. 
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4.4  OneSAF  Scenario  Implementation 


4.4.1.  Terrain  and  Environment 

The  Southwest  Asia  (SWA)  terrain  database  is  used  with  the  OneSAF  default 
values  as  follows  in  Table  2: 

Table  2.  Environmental  Conditions 


Temperature 

21°  C 

Humidity 

35% 

Sunrise 

0200  GMT 

Sunset 

1800  GMT 

Moonrise 

1800  GMT 

Moonset 

0200  GMT 

Visibility 

100% 

Cloud  Cover 

2% 

Wind  Speed 

7  knots\hr  from  the  north 

4.4.2.  Units  and  Task  Organization 

OneSAF  entities  and  units  selected  to  represent  the  Coalition  and  Insurgent  forces 
are  shown  in  Table  3. 

Table  3.  OneSAF  Entities  and  Units 


Unit  or 
Entity 

Description 

Initial  Position 

OneSAF  Filename 

Coalition  Forces 

1st  Platoon 
Stryker 

Recon 

Includes  four  Stryker  vehicles, 
Light  and  Heavy  sections, 
each  Stryker  includes 

33.31505°  N 

44.33443°  E 

.../mr/SBCT_UNIT/PL 
T/PLT  Stryker  recon 
WithDismountsSBC 
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individual  combatant 
dismount  team. 

TInfBnU  S  .xml 

UGV 

Specified  in  ANVEL,  but  use 
a  OneSAF  HMMWV  for 
scenario  testing  without 
ANVEL. 

33.32257°  N 

44.31 180°  E 

Movement  controlled 
by  ANVEL 

AH-64A-1 1 
Apaches 

Consists  of  five  AH-64A-1 1 
aircraft,  only  two  are  assigned 
tasks. 

33.26222°  N 

44.24556°  E 

../lr/UA  VN  UNIT/PL 
T/PLT  ATK  ACFT  A 
H64A  ATK  RECON 
CO  RWA  US.xml 

Insurgent  Forces 

Insurgent 

Squad 

Mortar  Squad  with  1 82-mm 
Mortar  Team,  and  GAZ66 
Truck.  The  truck  is  manually 
destroyed  during  the  scenario 
definition. 

33.22456°  N 

44.31028°  E 

../mr/COMBAT/INFA 
NTRY/SQD/SQD  Gue 
rrilla_Mortar_OPF  OR. 
xml 

4.4.3.  OneSAF  Tasks 

Table  4  shows  the  OneSAF  tasks,  purposes,  and  order  of  execution  within  the 
scenario. 


Table  4.  OneSAF  tasks. 


Execution 

Sequence 

Unit  or  Entity 

Task  Names 

Purpose 

Order  of 
Execution 

1 

Insurgent: 

ASST 

GUNNER- 

IC3 

a)  Move  Tactically 

b)  Emplace 
Explosives 

c)  Move  Tactically 

Move  to  location; 
emplace  an  IED  (203- 
mm,  HE-FRAG  with  a 
proximity  fuse)  at 
33.322697°  N, 
44.31081°  E.  Return 
to  origin. 

Upon 

execution  of 
OneSAF 
(start  of  run) 
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Execution 

Sequence 

Unit  or  Entity 

Task  Names 

Purpose 

Order  of 
Execution 

2 

Insurgent: 

MTR  TM 
LDR-IC1 

a)  Move  Tactically 

b)  Emplace 
Explosives 

c)  Move  Tactically 

Move  to  location; 
emplace  an  IED  (105- 
mm,  HE  M760  round 
using  a  trip  wire)  at 
33.32273°  N, 

44.31075°  E.  Return 
to  origin. 

After  (1)  is 
completed. 

3 

Coalition: 

Stryker 

Platoon 

After  IC  mount, 
follow  route  to  final 
leg  (Figure  5). 

Simulates  4-km  patrol 
through  an  urban  area, 
prior  to  use  of  UGV 
for  local 
reconnaissance. 

After  (1)  is 
initiated. 

4 

HMMWV1 

Tactical  Move. 

Lead  patrol  through 
urban  area,  then 
follow  last  leg  into 
area  of  IED. 

Prior  to  (3). 

1  Used  only  for  testing  scenario  execution  without  ANVEL. 

4.4.4.  OneSAF/ANVEL  Scenario  Execution  and  Lessons  Learned 

In  order  for  the  OneSAF  scenario  to  execute  as  envisioned  and  be  used  as  an 
analytical  tool  and  reduce  the  amount  of  human  intervention,  a  number  of  new  or 
modified  OneSAF  behavior  tasks  and  capabilities  need  to  be  developed.  Note  that  these 
observations  are  based  on  OneSAF  v2.1  and  some  of  these  maybe  overcome  by  user 
developed  behaviors  using  the  OneSAF  behavior  development  tool,  and  future  OneSAF 
releases. 

•  Ground  vehicle  movement  in  urban  terrain  along  streets  is  a  known  OneSAF  problem; 
route  planning  near  buildings,  and  sharp  turns  are  difficult,  and  which  need  to  be 
improved  before  this  scenario  can  be  used  in  a  fully  automated  manner. 
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•  Movement  of  the  Stryker  unit  into  and  out  of  defensive  positions/formation  in  the 
scenario  (while  waiting  for  UGV  to  conduct  recon)  was  not  possible  due  to  urban  route 
planning  issues  already  mentioned  above. 

•  Communication/situational  analysis  between  entities  in  the  same  OneSAF  unit  works 
well,  but  allowing  reports  from  another  unit  did  not  seem  possible,  nor  is  it  yet  possible  to 
obtain  reports  from  vehicles  represented  in  ANVEL  using  MATREX.  Although  the  issue 
of  SALUTE  reports  and  HLA  has  been  discussed  in  the  OneSAF  user  Forum. 

•  IED  which  can  be  created  in  OneSAF  did  not  cause  damage  to  the  activating  entity  (this 
problem  was  reported  to  the  OneSAF  help  desk). 

•  While  convoys  can  detect  area  minefields  and  react,  there  currently  is  no  behavior  which 
allows  units  to  detect  IED  and  report  or  react  to  them. 


Chapter  5:  Assessing  the  UGV  Testbed  Engineering-Level  Experiment 

5.1  Measures  of  Effectiveness 

In  order  to  assess,  quantify  and  communicate  how  well  the  testbed  environment 
performs,  several  measures  of  effectiveness  (MOE)  are  considered  in  the  following  areas: 
integration  and  interfaces  to  other  models,  run-time  performance,  fidelity,  resolution, 
incorporation  of  human-in-the-loop  (HITL)  operation,  graphics\scene  quality,  mission 
design  to  include  the  mission  tasks  and  end-to-end  test  coverage,  and  the  ease  of 
development.  Each  of  these  will  be  detailed  in  this  chapter. 
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5.1.1.  Interfaces  and  Integration  with  Other  Models 


The  testbed  consists  of  ANVEL  and  OneSAF  and  integrates  over  the  MATREX 
run-time  interface  (RTI)  which  facilitates  the  integration  of  multi-fidelity  models  and 
HITL  interactions  in  an  HLA  environment  as  recommend  in  Goerger  et  al.  (2008).  Using 
this  architecture  will  also  allow  the  models  within  the  testbed  to  potentially  interface 
more  easily  to  other  models.  Future  experiments  will  look  at  incorporating  higher  fidelity 
sensor  models  from  Night  Vision  Laboratory  Toolkit  and  battle  command  and  control 
from  the  Battle  Command  Management  System.  A  MOE  associated  with  this  is  the 
actual  integration  test  coverage.  This  measures  the  fraction  of  the  system  that  has 
undergone  end-to-end  testing  satisfactorily.  Another  MOE  considered  is  the  complexity 
of  interfaces  and  integration.  The  testbed  environment  needs  to  be  capable  of  being 
replicated  with  the  expected  amount  of  effort.  Component  interface  and  middleware  or 
integration  code  complexity  should  not  be  above  and  beyond  what  is  currently  expected 
given  similar  integration  efforts  done,  i.e.,  BLCSE  federation  integration.  Finally,  the 
testbed  should  have  adaptability  thus  allowing  for  the  integrated  system’s  ability  to  adapt 
to  requirement  changes. 

5.1.2.  Runtime  Performance 

The  runtime  performance  of  the  testbed  is  near-real-time.  The  capabilities  that 
exist  within  the  testbed  now  as  well  as  functionality  that  will  be  added  in  the  future  and 
continue  to  allow  the  testbed  to  perform  at  near-real  time  runtime  performance  are 
summarized  below  in  Table  5.  Runtime  performance  capabilities  timeline. 


19 


Table  5.  Runtime  performance  capabilities  timeline. 


Now 

Near-Term  <1  yr 

Mid-Term  1-2 
yrs 

Long-Term  3-5 
yrs 

UGV  mobility 

Improved 

collision 

detection 

Improved 

mobility 

High  Fidelity  \IR 
Sensors  and 
Cameras 

Optical  Visual 
Cameras 

Use  of  OneSAF 

sensors 

Additional 
sensor  models 

HPC  Environment 
(Not  H LA 
Compliant) 

SAF 

Environment 

Leverage 

OneSAF 

capabilities 

Integrate 

additional 

models 

UGV  functionality 
incorporated  into 
OneSAF 

HITLUJGV 

Interaction 

Add  C2/SA 
models 

Semi- 

autonomous 

Autonomous  UGV 

5.1.3.  Fidelity 


The  capabilities  that  exist  within  the  testbed  now,  as  well  as  functionality  that  will 
be  added  in  the  future,  are  summarized  below  in  Table  6.  Fidelity  capabilities  timeline. 


Table  6.  Fidelity  capabilities  timeline. 


HITL  guided  Mobility 
and  Recon 


HITL  Search  and 
Optical  Visual 
Cameras 


HITL  Detect 


HITL  Locate 


Interaction  with 
objects  within  AN  VEL 


Near-Term  <1  yr 

Mid-Term  1-2  yrs 

Long-Term  3  -  5  yrs 

Enhanced  Collision 
Detection\Assisted 
Tele-OP 

Waypoint  navigation 
with  operator  inputs 

Soil  deformation 

OneSAF  sensor 
capabilities  (IR) 

added  sensor 

capabilities 

Semi-autonomous 

Autonomous  Search 
HPC  High  Res 
Environment  (Not 

HLA  Compliant) 

Integration  of 
BCMS\NVL  tools 

Semi-autonomous 
Communication 
System  Reporting 

Autonomous 

Detection 

Improved  C2/SA 

Communication 
System  Reporting 

One  way  interaction 
ANVEL  OneSAF 

Limited  two  way 
interaction  ANVEL 
OneSAF 

Two  way  interaction 
ANVEL  <-->  OneSAF 
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5.1.4.  Resolution 


Resolution  is  the  measure  of  how  much  the  model  looks  like  the  real  thing.  The 
resolution  of  the  ANVEL  model  can  be  seen  in  Figure  6.  ANVEL  is  capable  of 
displaying  ultra  high  resolution  LIDAR  data.  The  LIDAR  data  currently  being  used  is 
lcm  resolution.  OneSAF  is  capable  of  displaying  low-resolution  polygons  with  20-  to 
30-cm  resolutions. 


-r  V  ,  r.  i  rr 

—  y  st 

1 

7  j  xt 

— '  IED -J  - 

PL-IED1 

Figure  6.  ANVEF  (Feft)  and  OneSAF  (Right)  testbed  resolution. 

OneSAF  resolution  is  summarized  as  follows  in  Table  7.  OneSAF  Resolution 
(OneSAF  Public  Site,  2008): 


Table  7.  OneSAF  Resolution 


Low 

Medium 

High 

Very  High 

Ultra  High 

1:1M 

1:250K 

1:50K 

1:12K 

Buildings 

5.1.5.  Incorporation  of  Human-in-the-Loop  (HITL) 

HITE  tele-operation  via  keyboard  or  joystick  operation  moves  the  UGV  within 
the  ANVEF  and  integrated  OneSAF  model  simultaneously.  The  HITE  also  facilitates 
message  passing  and  situational  awareness  updates  to  the  common  operating  picture 
(COP). 
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5.1.6.  Graphics/Scene  Quality 


Figure  7  is  a  high  quality  picture  comparable  to  high  end  gaming  graphics. 


Figure  7.  ANVEL  scene  quality. 

5.1.7.  Mission  Design 

The  UGV  testbed  should  be  able  to  accomplish  the  Universal  Joint  Task  List 
(UJTL)  (2008)  and  the  Army  Universal  Task  List  (AUTL)  (2003)  tasks  associated  with 
UGV.  The  UJTL  Tasks  identified  are  summarized  in  Table  8.  Identified  UJTL  Tasks 
below: 


Table  8.  Identified  UJTL  Tasks 


OP  1 

Conduct  Operational  Movement  and 
Maneuver 

OP  1.5.1 

Control  of  operationally  significant  land  area 
in  the  joint  operations  area 

OP  2 

Provide  Operational  ISR 

OP  2.2 

Collect  and  Share  Operational  Information 

OP2.2.1 

Collect  Information  on  Operational  Situation 
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OP  2.2.5 


Collect  Target  Information 


The  AUTL  tasks  identified  are  summarized  in  Table  9.  Identified  AIJTL  tasks 

below: 


Table  9.  Identified  AUTL  tasks 


Art.  1.0 

The  Movement  and  Maneuver  War  fighting 
Function 

Art.  1.2 

Conduct  Tactical  Maneuver 

Art  1.2.5 

Exploit  Terrain  to  Expedite  Tactical 
Movements 

Art  1.2.10 

Navigate  From  One  Point  to  Another 

Artl.6 

Conduct  Mobility  Operations 

Art.  1.6.1 

Overcome  Barriers,  Obstacle,  and  Mines 

Art  1.6.2 

Enhance  Movement  and  Maneuver 

Art  1.6.3 

Negotiate  a  Tactical  Area  of  Operations 

Art  2.0 

The  Intelligence  War  fighting  Function 

Art  2.2 

Support  to  Situational  Understanding 

Art.  2.2.3 

Provide  Intelligence  Support  to  Protection 

Art.  2.3 

Conduct  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR) 
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Art.  2.3.3 

Conduct  Tactical  Reconnaissance 

Art.  2.3.4 

Conduct  Surveillance 

Art  2.4 

Provide  Intelligence  Support  to  Targeting 
and  Information  Operations  Capabilities 

Art  2.4.1 

Provide  Intelligence  Support  to  Targeting 

Art.  2.4.2 

Provide  Intelligence  Support  to  Information 
Operations  Capabilities 

5.1.8.  End-to-End  Test  Coverage  (Verification  and  Validation) 

As  the  testbed  environment  grows  to  encompass  additional  models,  the  end-to- 
end  test  coverage  will  grow  in  complexity.  It  is  desired  that  100%  end-to-end  test 
coverage  will  be  completed  and  it  will  be  important  to  document  the  fraction  of  the 
systems  functionality  that  has  undergone  end-to-end  testing  satisfactorily.  This  process  is 
known  as  verification  and  validation  of  the  testbed  environment.  As  tests  are  completed 
it  is  especially  important  to  complete  the  documentation  of  known  limitations  and  work¬ 
arounds  in  order  to  inform  about  the  actual  functionality  capable  within  the  model.  The 
actual  thread  tests  conducted  for  Experiment  2  are  discussed  in  great  detail  in  section  5.3. 

5.1.9.  Ease  of  Development 

It  is  important  that  the  testbed  that  is  developed  is  not  so  overly  complex  that  it 
requires  a  highly  skilled  group  of  operators  to  recreate  or  use  the  environment.  The 
development  effort  therefore  concentrated  on  tools  available  to  all  stakeholders  that  do 
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not  require  an  extended  effort  beyond  what  the  stakeholders  would  need  to  operate  this 
environment. 

5.2  Design  of  Experiment 

In  conducting  an  experiment  of  the  actual  testbed  itself,  factors  to  be  considered 
while  assessing  the  environment  are  the  1)  Sensors\Cameras  with  regard  to  the  optical 
resolution,  black  and  white  versus  color,  contrast,  camera  field-of-view  (FOV),  and  2)  the 
load  on  the  federation  such  as  the  number  of  HITLYWorkstations  federated.  These  factors 
may  have  an  effect  on  the  measures  of  effectiveness  as  outlined  in  Section  5.1. 

5.3  Verification  and  Validation 

The  verification  and  validation  (V&V)  of  the  federation  is  the  most  important  part 
of  assessing  the  UGV  testbed.  Verification  is  the  process  of  determining  if  the  modeling 
and  simulation  (M&S)  represents  the  conceptual  description  of  the  UGV  testbed. 
Validation  is  the  process  of  determining  to  what  extent  the  M&S  adequately  represents 
the  real  world  functionality  of  the  UGV  in  its  real  environment.  The  V&V  is 
accomplished  by  identifying  the  mission  threads  that  need  to  be  assessed  and  then 
following  these  mission  threads  through  all  affected  M&S.  The  mission  threads  for  Use 
Case  #1  were  identified  as  the  maneuver,  search,  detect,  and  locate  threads.  The 
operational  and  technical  views  of  these  threads  were  developed  and  are  discussed  in  the 
following  two  sections.  The  operational  perspective  is  the  actual  practical  application 
within  the  testbed  environment  while  the  technical  perspective  is  the  M&S  application 
within  the  testbed  environment. 
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5.3.1.  Maneuver  Thread 


The  maneuver  thread  examines  the  UGV  maneuver  in  ANVEL  as  well  as  in 
OneSAF.  The  UGV  should  move  over  the  same  route  in  each  model  and  should  correlate 
to  the  same  terrain  location  in  each  model  at  every  instance.  As  the  UGV  platform  is 
maneuvered  by  the  HITL  operator,  the  information  on  the  UGV  entities  location  should 
update  in  real  time  in  OneSAF.  The  location  of  the  entity  on  the  terrain  should  correlate 
with  the  location  and  terrain  in  ANVEF.  Figure  8  and  Figure  9  show  the  maneuver 
thread  operational  and  technical  perspective,  respectively.  The  operational  perspective  of 
maneuver  is  described  as  follows.  The  route  is  planned  for  the  entities  to  execute.  The 
entities  then  traverse  the  terrain  maneuvering  the  terrain  and  obstacles  it  encounters  them. 
As  the  UGV  encounters  obstacles  and  reacts  the  UGVs  location  information  is  transferred 
to  the  other  entities  within  the  scenario. 


UGV  Locati 
and  Terrain 
C  Of  relation 


Produce, 
Dis  seminal* 
UGV  Location 
Information: 


Maneuver  Terrain 
and  Obstacles 


Receive  UGV 


Figure  8.  Maneuver  thread  operational  perspective. 


26 


The  technical  perspective  of  maneuver  is  described  as  follows.  The  HITL 
controls  and  executes  the  UGVs  maneuver  as  per  the  planned  route  by  maneuvering  the 
UGY  within  ANVEL.  ANVEL  displays  the  UGV  in  the  high  resolution  environment  and 
passes  the  UGVs  current  location  information  across  the  MATREX  RTI  to  update  the 
OneSAF  COP  of  the  UGVs  actual  location  within  the  terrain. 


Figure  9.  Maneuver  thread  technical  perspective. 


5.3.2.  Search  Thread 

The  search  thread  examines  sensor\camera  search  functions  within  the  testbed 
environment.  Figure  10  and  Figure  1 1  show  the  search  thread  operational  and  technical 
perspective,  respectively.  The  operational  perspective  of  search  is  as  follows.  Given  the 
planned  route,  as  the  UGV  moves  along  the  planned  route,  the  HITE  moves  the  cameras 
within  ANVEE  to  search  the  terrain. 
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Figure  10.  Search  thread  operational  perspective. 


The  technical  perspective  for  search  is  described  as  follows.  The  HITL  controls 
the  UGVs  movement  within  ANVEL  along  the  planned  route  as  well  as  the  movement  of 
the  cameras  to  search  the  environment. 
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5.3.3.  Detect  Thread 


The  detect  thread  examines  sensor\camera  detect  functions  within  the  testbed 
environment.  Figure  12  and  Figure  13  show  the  detect  thread  operational  and  technical 
perspective,  respectively.  The  operational  perspective  is  described  as  follows.  The 
deployed  camera  allows  the  HITL  to  view  objects  along  the  planned  route  and  the  HITL 
uses  judgment  to  determine  if  an  object  that  is  detected  is  an  IED  and  should  be  reported 
higher. 


Information  Requirements 


Exploit 
Intelligence: 
Make  Decision 
Deploy  sensors 


Interaction  of 
All  Federate  Members 
Includes  human  in  the  loop 
data  input. 

analysis  and  evaluation 
of  information, 
and  exploitation 


Produce. 

Disseminate 

Information: 


Receive  Target 

Information: 

Fuse 

Analyze 

Evaluate 


Figure  12.  Detect  thread  operational  perspective. 


The  technical  perspective  is  described  as  follows.  The  HITL  through  viewing  the  camera 
view  within  ANVEL  determines  if  an  object  is  suspicious  and  warrants  reporting  to  the 
COP.  The  HITL  reports  to  the  HITL  controlling  the  COP  to  inform  the  entities  within 
OneSAL  that  the  IED  has  been  detected. 
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Figure  13.  Detect  thread  technical  perspective. 


5.3.4.  Locate  Thread 

The  locate  thread  examines  sensor\camera  search  functions  within  the  testbed 
environment.  Figure  10,  Figure  14,  and  Figure  15  show  the  locate  thread  operational  and 
technical  perspective,  respectively.  The  operational  perspective  is  described  as  follows. 
Once  the  object  has  been  detected  to  be  an  IED,  the  HITL  determines  the  objects  grid 
coordinates  and  the  HITL  reports  the  grid  location  to  the  HITL  operator  controlling  the 
COP. 
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Information  Requirements 


-J _ ■ 


Locate  Target 
in  Terrain 


Target  Location 
and  Terrain 
Correlation 


Interaction  of 
All  Federate  Members 
Includes  human  in  the  loop 
data  input, 

analysis  and  evaluation 
of  information, 
and  exploitation 


Produce. 

Disseminate 

Target 

Location 

Information: 


Receive  Target 

Location 

Information: 

Fuse 

Analyze 

Evaluate 


Figure  14.  Locate  thread  operational  perspective. 


The  technical  perspective  is  described  as  follows.  The  HITL  determines  the  grid 
coordinate  location  of  the  IED  and  reports  the  information  to  the  HITL  operating  the 
COP.  The  HITL  operating  the  COP  then  directs  the  OneSAF  entities  to  react  to  the  IED 
at  the  given  location. 


Figure  15.  Locate  thread  technical  perspective. 
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5.4  UGV  Testbed  Environment  Experimental  Results 


The  testbed  has  shown  that  ANVEL  can  interface  and  is  integrated  with  OneSAF 
through  MATREX.  Given  the  role  of  MATREX  within  this  federation,  additional 
models  should  be  able  to  fully  interface  and  be  integrated  into  the  federation  as  additional 
M&S  functionality  is  required. 

The  ANVEL\OneSAF\MATREX  federation  runtime  performs  at  near-real  time. 
Section  5.1.2  details  additional  functionality  and  when  near-real-time  runtime 
performance  can  be  expected. 

The  modeling  fidelity  within  the  testbed  relies  heavily  on  HITL  interactions 
especially  with  respect  to  situational  awareness.  Section  5.1.3  details  the  current  fidelity 
of  the  M&S  as  well  as  projects  future  fidelity  capabilities. 

This  testbed  is  a  multiple  resolution  environment  with  ANVEL  modeling  the  high 
resolution  UGV  environment  down  to  1-cm  resolution. 

This  UGV  testbed  incorporates  HITL  and  relies  on  the  HITL  to  perform  situational 
awareness  and  command  and  control  functionality.  Section  5.1.3  projects  when  semi- 
and  autonomous  functionality  can  be  anticipated. 

The  graphics\scene  quality  is  comparable  to  graphics  within  the  gaming  industry 
without  sacrificing  the  physics-based  modeling. 

The  mission  design  within  the  UGV  testbed  is  capable  of  performing  AUTL  and 
UJTL  tasks.  The  tasks  for  Experiment  2  are  those  associated  with  Use  Case  #1  already 
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detailed  in  Chapter  4  and  employ  the  threads  of  maneuver,  search,  detect,  and  locate. 
The  UJTL  tasks  are  described  in  Table  8  and  the  AUTL  tasks  are  described  in  Table  9. 

One  hundred  percent  end-to-end  test  coverage  was  completed  on  the  UGV  testbed 
environment.  The  V&V  of  the  maneuver,  search,  detect,  and  locate  threads  conducted  on 
the  UGV  testbed  resulted  in  the  following  federation  known  limitations  of  no 
autonomous  message  passing  (search\detect\locate).  The  work-arounds  established  to 
combat  this  limitation  is  to  utilize  HITL  message  passing. 

Making  use  of  OneSAF  and  the  MATREX  environment  greatly  assists  in  the  ease 
of  development  of  this  federation  testbed.  The  MATREX  RTI  and  OneSAF  are  currently 
used  in  the  BLCSE  and,  therefore,  already  have  a  base  support  of  users  and  technical 
experience. 


Chapter  6:  Assessing  the  UGV  Testbed  Operational-Level  Experiment 

6.1  FCS  UGV  Performance  Metrics 

In  assessing  the  actual  UGV  performance,  the  following  FCS  UGV  performance 
metrics  were  noted.  These  performance  metrics  are  where  we  chose  the  UGV 
performance  metrics  for  assessing  the  UGV  in  the  testbed  with  Use  Case  #1.  The  FCS 
UGV  performance  metrics  are  summarized  in  Table  10.  FCS  UGV  performance  metrics 
below: 
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Table  10.  FCS  UGV  performance  metrics 


Primary 

Secondary 

Endurance 

Air  Drop-ability 

Mobility 

Robustness  to  Crashes 

Payload  Fraction 

Reliability 

Signature 

Cost 

6.2  Use  Case  #1  Measures  of  Effectiveness 

Considering  the  FCS  performance  metrics,  we  determined  the  Use  Case  #1 
measures  of  effectiveness  (MOE).  We  selected  the  following  MOEs  as  outlines  in  Table 
11.  Experiment  2  MOE’s  to  focus  on  for  Experiment  2: 
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Table  11.  Experiment  2  MOE’s 


Endurance 

Time 

Mobility 

Urban 

Speed 

Rough 

Speed 

Open 

Speed 

Reliability 

Probability  of  failure  free  system  over 

specified  period  of  time 

Exposure  to  threat 

%  of  Time  Lost  Communications 

6.3  Design  of  Experiment 

In  conducting  an  experiment  of  the  UGV  within  the  testbed,  factors  to  be 
considered  while  assessing  the  UGV  measures  of  effectiveness  are  the  1)  Terrain  Type, 
2)  UGV  Speed,  3)  Time  to  mission  accomplishment,  and  4)  Type  of  Platform,  5) 
Sensors\Cameras  with  regard  to  the  optical  resolution,  black  and  white  versus  color, 
contrast,  and  camera  FOV.  These  factors  could  have  an  effect  on  the  measures  of 
effectiveness  as  outlined  in  Table  11.  Experiment  2  MOE’s  above. 
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Chapter  7:  Summary 


High  fidelity  environments  are  needed  to  impact  UGV  performance  in  synthetic 
environments  (e.g.,  navigation,  sensing,  and  movement).  Based  on  stakeholder  input,  we 
have  developed  a  UGV  HFSE  testbed  for  evaluating  UGV  design  alternatives.  The 
OneSAF\ANVEL  Federation  integrated  with  MATREX  is  conducive  as  a  HFSE  for 
UGV  testing  and  analysis  and  has  great  potential  for  use  within  other  federations. 


Chapter  8:  Future  Work 

FY09  work  will  integrate  ANVEL  with  battle  command  tools  while  exploring 
other  use  cases  as  outlined  in  Nagle  et  al., (2008).  The  addition  of  Command,  Control  and 
Communications  (C3)  Situational  Awareness  models  will  be  explored  so  there  is  less 
reliance  on  the  HITL  to  provide  all  SA  on  the  UGV  actions.  This  work  may  be  extended 
to  conduct  virtual  testing  of  red  team  robotics  in  the  UGV  testbed,  to  examine  unmanned 
aerial  systems  (UAS)  interoperability  within  this  environment,  to  conduct  UGV 
component  optimization  as  well  as  to  add  additional  UGV  functionality\components  into 
the  UGV  testbed. 

The  priority  of  work  for  future  M&S  enhancements  to  ANVEL  are  collision 
detection,  damage  assessment,  autonomous  controller  and  C3\SA  work  to  facilitate  two 
way  communications  between  the  models. 
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Appendix  A:  Acronyms 


A 

ACM 

Association  for  Computing  Machinery 

AH 

Attack  Helicopter 

ANS 

Autonomous  Navigation  System 

ANVEL 

Autonomous  Navigation  Virtual  Environment  Laboratory 

AO 

Area  of  Operation 

AT\AP 

Anti  tan  k\  A  n  ti  p  ersonn  e  1 

ART 

Army  Tactical  Tasks 

AUTL 

Army  Universal  Task  List 

B 

BFA 

Battlefield  Functional  Area 

BLCSE 

Battle  Lab  Collaborative  Simulation  Environment 

BLOS 

Beyond  Line  of  Sight 

C 

C3 

Command,  Control,  Communication 

C4I 

Command,  Control,  Communication,  Computers, 

Intelligence 

CBRN 

Chemical,  Biological,  Radiological,  and  Nuclear 

CBRNE 

Chemical,  Biological,  Radiological,  Nuclear,  and  High- 
Yield  Explosive 

CG 

Commanding  General 

CGF 

Computer  Generated  Forces 

COA 

Course  of  Action 

COP 

Common  Operating  Picture 
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D 

DIS 

Distributed  Interactive  Simulation 

DoD 

Department  of  Defense 

E 

ERDC 

U.S.  Army  Engineer  Research  and  Development  Center 

ERDC  -CRREL 

Engineer  Research  and  Development  Center  -  Cold  Regions 
Research  Lab 

ERDC-GSL 

Engineer  Research  and  Development  Center  -  Geotechnical 
and  Structures  Laboratory 

F 

FCS 

Future  Combat  Systems 

FOC 

Force  Operating  Capabilities 

FOV 

Field  of  View 

G 

GMT 

Greenwich  Mean  Time 

GRMP 

Ground  Robotics  Master  Plan 

H 

HFSE 

High-Fidelity  Synthetic  Environment 

HITL 

Human  in  the  Loop 

HLA 

High  Level  Architecture 

HMMWV 

High  Mobility  Multipurpose  Wheeled  Vehicle 

I 

IEEE 

Institute  of  Electrical  and  Electronics  Engineers 

IED 

Improvised  Explosive  Device 

IR 

Infrared 
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ISR 

Intelligence,  Surveillance  and  Reconnaissance 

J 

JAUS 

Joint  Architecture  for  Unmanned  Systems 

JGRE 

Joint  Ground  Robotics  Enterprise 

L 

LIDAR 

Light  Detection  and  Ranging 

LOS 

Line  of  Sight 

M 

M&S 

Modeling  and  Simulation 

MANSCEN 

Maneuver  Support  Center 

MARCORSYSCOM 

Marine  Corps  Systems  Command 

MATREX 

Modeling  Architecture  for  Technology,  Research  and 
Experimentation 

MCT 

Marine  Corps  Tasks 

MCTL 

Marine  Corps  Task  List 

MEDEVAC 

Medical  Evacuation 

MEVA 

Mission  Essential  Vulnerable  Area 

MOE 

Measure  of  Effectiveness 

MOUT 

Military  Operations  in  Urban  Terrain 

N 

NLOS 

Non-line  of  Sight 

NLT 

No  later  than 

O 

OneSAF 

One  Semi-Automated  Forces 

ORCEN 

Operations  Research  Center  of  Excellence 
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OSD 

Office  of  the  Secretary  of  Defense 

P 

PEO-GCS 

U.S.  Army  Program  Executive  Office  Ground  Command 
Systems 

R 

RDECOM 

Research,  Development,  and  Engineering  Command 

RDT&E 

Research,  Development,  Test  and  Evaluation 

RS  JPO 

Robotic  Systems  Joint  Project  Office 

RTI 

Runtime  Interface 

RWA 

Rotary  Wing  Aircraft 

S 

S&T 

Science  and  Technology 

SA 

Situational  Awareness 

SALUTE 

Size,  Activities,  Location,  unit  identification,  Time  and 

Date,  Equipment 

SBCT 

Stryker  Brigade  Combat  Team 

SDP 

Systems  Decision  Process 

SQD 

Squad 

SWA 

Southwest  Asia 

T 

TENA 

Test  and  Training  Enabling  Architecture 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

U 

UAS 

Unmanned  Aerial  System 

UAV 

Unmanned  Aerial  Vehicle 
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UGV 

Unmanned  Ground  Vehicle 

UJTL 

Universal  Joint  Task  List 

UMS 

Unmanned  Systems 

UNTL 

Universal  Naval  Task  List 

USMA 

United  States  Military  Academy 

V 

v&v 

Verification  and  Validation 

VANE 

Virtual  Autonomous  Navigation  Environment 
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Appendix  B:  Stakeholder  Analysis  Recommendations 


Findings 

Conclusions 

Recommendations 

Desire  for  HFSE  to  readily  integrate 
through  interfaces  with  other  models, 
simulations,  and  systems. 

Use  existing  policies,  directives, 
and  best  practices  that  address 
standards  and  open 
architectures  to  minimize  re¬ 
engineering. 

HFSE  should  conform  to  data, 
data  interchange,  information 
exchange  and  communications 
standards  including  JAUS. 

Requirement  (2013)  for  a  Common  Open 
Architecture. 

Desire  to  incorporate  simulated 
and  real  systems  such  as 
control  units. 

HFSE  should  haveHLAor  DIS 
compliant  external  interfaces. 

JGRE  mandates  JAUS  for  all  its 
programs. 

Desire  for  human-in-the-loop 
interactive  mode  as  well  as  other 

modes. 

Investigate  Common  Open 
Architecture  requirement  and  its 
impact  on  HFSE  users. 

Need  for  near-real  time  or  faster 
computational  performance. 

Desire  to  run  simulations  in  near- 
real  time  or  faster. 

Must  have  near  real-time  for 
certain  applications  and  determin 
areas  where  feasible  in  near-,  mid 
,  and  long-term. 

Higher  fidelity  terrain  and  models  will  add 
value  to  modeling  UGV  systems  and 
subsystems  performance. 

Level  of  fidelity  and  resolution 
needed  for  terrain/environment 
and  models  dependent  on  study 
issues  and  componenets. 

HFSE  should  have  multi¬ 
resolution  capabilities. 

Need  for  dynamic  terrain,  weather,  and 
obscurant  representation. 

Desire  dynamic  terrain. 

Want  3D  visualization  for  humans-in-the- 
loop  simulation  to  look  very  realistic. 

Desire  realistic  scene  generation 
or  image  rendering. 

HFSE  must  include  3D 
visualization  for  multiple  display 
units. 

Modeling  communications  between  UGV 
platforms  (current  and  future)  and 
controllers  or  other  systems  is  important. 

Incorporate  terrain  features  and 
attributes  so  as  to  impact 
LOS/NLOS/BLOS 
communications. 

Investigate  communication 
algorithms  used  by  different 
programs  and  determine  impacts 
on  environmental  content  and 
representation. 

The  most  frequently  cited  gaps  identified 
in  the  GRMP  were  associated  with 
maneuver,  search,  detect,  and  locate 
tasks. 
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